Statistical Mechanics of Multiplex Ensembles: Entropy and Overlap 
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There is growing interest in multiplex networks where individual nodes take part in several layers 
of networks simultaneously. This is the case for example in social networks where each individual 
node has different kind of social ties or transportation systems where each location is connected to 
another location by different types of transport. Many of these multiplex are characterized by a 
significant overlap of the links in different layers. In this paper we introduce a statistical mechanics 
framework to describe multiplex ensembles. A multiplex is a system formed by N nodes and M 
layers of interactions where each node belongs to the M layers at the same time. Each layer a 
is formed by a network G a . Here we introduce the concept of correlated multiplex ensembles in 
which the existence of a link in one layer is correlated with the existence of a link in another layer. 
This implies that a typical multiplex of the ensemble can have a significant overlap of the links in 
the different layers. Moreover we characterize microcanonical and canonical multiplex ensembles 
satisfying respectively hard and soft constraints and we discuss how to construct multiplex in these 
ensembles. Finally we provide the expression for the entropy of these ensembles that can be useful 
to address different inference problems involving multiplexes. 
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I. INTRODUCTION 



In the lastyears large attention has been paid to single 
networks [lH^ with breakthroughs revealing the deep re- 
lation between topological properties of the networks and 
their dynamics faQ. Nevertheless, many systems are not 
formed by isolated networks, instead they are formed by a 
network of networks ■ Examples include multimodal 
transportation networks [HI El, climatic systems [Hj], 
economic markets [l3j. energy-supply networks (l4j and 
the human brain |l5[ . Moreover many networks are mul- 
tiplex indicating the fact that two nodes can belong to 
different networks at the same time. For example this is 
the case of social networks in which agents can be linked 
at same time, by familiar relationships, friendship, pro- 
fessional collaboration, co-location, email communication 
and so on. The offshoot of the network theory fundamen- 
tal insights is that for us working in statistical mechanics 
it is now possible - in a sense it is mandatory to move into 
the field to shed light on the complexity on interdepen- 
dent networks and multiplexes. In this context, new mea- 
sures for multiplex T(| [l^ and new models of grow- 
ing multiplexes 01121 have been proposed. Moreover, 
several works have studied dynamical processes taking 
place on multiplexes and interacting networks and new 
surprising phenomena have been observed in this context 
involving percolation [l4 . [20rl22j , cascades p3| , diffusion 
f24j . epidemic spreading 25 1 and cooperati on!^ .o pinion 
dynamics (27j and community detection^. l28l. |29| . 

Yet, we are only at the beginning of the research on 
interacting networks and multiplexes and we need to de- 
velop further theoretical frameworks to extract informa- 
tion from multiplex data. For this purpose we need new 
statistical mechanics methods to analyze multiplex and 
interacting networks data. 

An important tool to study real networks is to com- 
pare them with null models represented by randomized 



network ensembles. For single networks an equilibrium 
statistical mechanics framework has been recently formu- 
lated [30l - l42l | in order to characterize network ensembles. 
A network ensemble is defined as a set of networks that 
satisfy a given number of structural constraints, i.e. de- 
gree sequence, community structure etc. Every set of 
constraints can give rise to a microcanonical network en- 
semble, satisfying the hard constraints, or to a canonical 
network ensemble in which the constraints are satisfied 
in average. This construction is symmetric to the clas- 
sical ensemble in statistical mechanics where one consid- 
ers system configurations compatible either with a fixed 
value of the energy (microcanonical ensembles) or with 
a fixed average of the energy determined by the thermal 
bath (canonical ensembles). For example the G(N,L) 
random graphs formed by networks of N nodes and L 
links is an example of microcanonical network ensemble 
while the G(N,p) ensembles, where each pair of links is 
connected with probability p, is an example of canonical 
network ensemble since the number of links can fluctuate 
but has a fixed average given by (L) = pN(N — l)/2. A 
theoretical question that arise in the study of network en- 
sembles is whether the microcanonical ensemble and the 
corresponding canonical ensemble are equivalent in the 
thermodynamics limit. It turns out [34l |35| that when 
the number of constraints in two conjugated network en- 
sembles is extensive, the ensembles are no longer equiv- 
alent in the thermodynamic limit and it is important to 
characterize their differences. For example microcanon- 
ical and canonical network ensembles with given degree 
sequence are non equivalent in the thermodynamic limit. 

The entropy of network ensembles is given by the log- 
arithm of the number of typical networks in the ensem- 
ble. The entropy of a network ensemble quantifies the 
complexity of the ensemble. In particular we have that 
the smaller is the entropy of the ensemble the smaller is 
the number of networks satisfying the corresponding con- 
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straints and implying that these networks are more op- 
timized. Both the network ensembles and their entropy 
can be used on several inferenceproblems to extract in- 
formation from a given network [43l [iij . Given the rele- 
vance of the statistical mechanics of randomized network 
ensembles for describing real networks, it is important 
to extend this successful approach to describe multiplex 
ensembles. In this paper we have chosen to consider only 
simple multiplex but the results can be easily extended 
to directed and weighted networks. We plan to consider 
these more complex cases in later publications. 

In this paper we will show how to treat multiplex en- 
sembles as null models for multiplexes. We will introduce 
a distinction between uncorrelated multiplex ensembles 
and correlated multiplex ensembles in which the exis- 
tence of a link in one layer is correlated to the existence 
of a link in another layer. We will characterize the over- 
lap between links in two different layers in the case of 
uncorrelated and correlated multiplex ensembles. We 
will evaluate the entropy of microcanonical and canoni- 
cal multiplex ensembles for a large variety of constraints. 
Finally this work open a new scenario for building null 
models of multiplex ensembles that has the promise to be 
used in a large variety of inference problems. The paper 
is organized as follows. In section II we introduce mul- 
tiplexes and the global and local overlap of the links in 
the two layers. In section III we introduce multiplex en- 
sembles, their entropy and correlations. In section IV we 
describe canonical multiplex ensembles, we distinguish 
these ensembles as correlated or uncorrelated. We give 
relevant examples of these ensembles, we calculate their 
entropy and their overlap and give algorithms to con- 
struct multiplexes in these ensembles. In section V we 
describe microcanonical multiplex ensembles. We give 
relevant examples of both correlated and uncorrelated 
microcanonical multiplex ensembles and calculate their 
entropy Finally in section VI we make the concluding 
remarks. 



II. MULTIPLEX AND OVERLAP BETWEEN 
TWO LAYERS 

Consider a multiplex formed by N labelled nodes i = 
1,2 ... ,N and M layers. We can represent the multiplex 
as described for example in Q. To this end we indicate 
by G = (G 1 , G 2 . . . G M ) the set of all the networks G a 
at layer a — 1,2, . . . , M forming the multiplex. Each 
of these networks has an adjacency matrix with matrix 
elements a" = 1 if there is a link between node i and node 
j in layer a = 1,2, ... M and zero otherwise. Moreover 
for a multiplex we can define multilinks, and multidegrees 
in the following way. Let us consider the vector fa = 
(mi, m<i, . . . , m a , ...ran) in which every element m a can 
take only two values m a =0,1. We define a multilink m 
the set of links connecting a given pair of nodes in the 
different layers of the multiplex and connecting them in 
the generic layer a only if m a = 1. We can therefore 



introduce the multiadjacency matrices A m with elements 
A?? equal to 1 if there is a multilink m between node i 
and node j and zero otherwise, i.e. the multiadjacency 
matrices have elements A™j = 0, 1 given by 

M 

A%=U[a%m a + (l-a?j)(l-m a )]. (1) 

Therefore we can define the total number of multilinks 
to in a network as the total number of pairs of nodes 
connected by a multilink m. Moreover we can define the 
multidegree to of a node i, k™ 1 as the total number of 
multilinks to incident to node i, i.e. 

N 

Af = £4r (2) 

We note here that the multilink m = between two 
nodes represent the situation in which in all the layers of 
the multiplex the two nodes are not directly linked. To 
have a uniform notation we refer also in this case to a 
multilink. Moreover we observe that the multiadjacency 
matrices are not all independent. In fact they satisfy the 
following normalization condition 

X># = 1. (3) 

in 

for every fixed pair of nodes (i,j). 

For two layers a, a' of the multiplex we can define the 
global overlap O a ' a as the total number of pair of nodes 
connected at the same time by a link in layer a and a 
link in layer a' , i.e. 

()—' X ( 4 ) 

i<j 

For a node i of the multiplex, we can define the local 
overlap o"' Q of the links in two layers a and a' as the 
total number of nodes j linked to the node i at the same 
time by a link in layer a and a link in layer a' , i.e. 

N 

°r' (5) 

We expect the global or the local overlap between two 
layers to characterize important correlations between the 
two layers in real-world situations. For example in a 
transportation multiplex, where the different layers can 
represent different kind of transport such as bus and train 
connections or private commuting, we expect that the 
links in the different layers of this multiplex have an 
overlap which is statistically significant respect to a null 
hypothesis of uncorrelation between the different layers. 
Also in social sciences if we consider the multiplex formed 
by different means of communication between people, 
(emails, mobile, sms, etc.) two people that are linked 
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in one layer are also likely to be linked in another layer, 
forming a multiplex of correlated networks. We note also 
that for two layer multiplex, i.e. M = 2 the multilink k\' 1 
is equal to the local overlap o^. Reversibly, the multide- 
gree kf 1 of a node i in a multiplex with generic number 
of layers M can be seen as a higher order local overlap. 



III. MULTIPLEX ENSEMBLES, ENTROPY 
AND CORRELATIONS 



A multiplex ensemble is specified when the probability 
P(G) for each possible multiplex is given. In a multiplex 
ensemble, if the probability of a multiplex is given by 
P(G), the entropy of the multiplex S is defined as 



S =-J2P(G) log P(G). 



IV. CANONICAL MULTIPLEX ENSEMBLES 
OR EXPONENTIAL RANDOM MULTIPLEXES 



The canonical multiplex ensembles are the set of mul- 
tiplex that satisfy a series of constraints in average. 

The construction of the canonical multiplex ensem- 
bles or exponential random multiplex follow closely the 
derivation or the exponential random graphs. 

We can build a canonical multiplex ensemble by max- 
imizing the entropy of the ensemble given by Eq. ((6]) 
under the condition that the soft constraints we want 
to impose are satisfied. We assume to have K of such 
constraints determined by the conditions 



(6) 



J2P(G)F, l (G)=C, 



(12) 



and measures the logarithm of the typical number of mul- 
tiplexes in the ensemble. As it occurs for single networks 
we can construct microcanonical or canonical multiplex 
ensembles according to the equilibrium statistical me- 
chanics approach applied to complex networks. More- 
over two layers in a multiplex network ensemble might 
be either correlated or uncorrelated. We will say that 
a multiplex ensemble is uncorrelated if the probability 
P{G) of the multiplex is factorizable into the probability 
of each single network G a in the layer a. Therefore in an 
uncorrelated multiplex ensemble we have 



M 



^(G)=n^( GQ )- 



(7) 



where P a (G a ) is the probability of network G a on layer 
a. If Eq. © doesn't hold,i.e. 



P(G)^l[P a (G a ). 



(8) 



we will say that the multiplex ensemble is correlated. 

Using Eq. (J7J we can show that the entropy of any 
uncorrelated multiplex ensemble is given by 

M M 

S = J^S a = -^P Q (G Q ) log P a (G a ), (9) 

a—l a—1 

where S a is the entropy of the network ensemble in layer 
a with probability P a (G a ). In an uncorrelated multi- 
plex the links in any two layer a and a' are uncorrelated 
therefore we have 



(a?<.) = <aS)<o«> 



(10) 



for every choice of pair of nodes 

On the contrary if the multiplex is correlated there will 
be at least two layers a and a' in a multiplex ensemble 
and a pair of nodes i and j for which 



(11) 



for fj, = 1, 2 . . . , K, where F fl (G) determines one of the 
structural constraints that we want to impose to the net- 
work. For example, F fJi (G) might characterize the total 
number of links in a layer of the multiplex G or the degree 
of a node in a layer of the multiplex G etc.. In the follow- 
ing we will specify in detail different major examples for 
the constraints F fl (G). In order the build the maximal 
entropy ensemble satisfying the soft constraints defined 
Eqs. (fl~2|) . we maximize the entropy S given by Eq. (|6]) 
under the condition that the ensemble satisfies the K 
soft constraints given by Eqs. (|12[) . Introducing the La- 
grangian multipliers A M enforcing the conditions given by 
Eqs. (fT2|) and the Lagrangian multiplier A enforcing the 
normalization of the probabilities P{G) = 1 we find 

the expression for the probability P(G) of a multiplex by 
solving the following system of equations, 



dP{G) 



S ~ E X » E MG)P{G) - A Y, P(G) 

H =1 G G 



= 0. 



(13) 



Therefore we get that the probability of a multiplex 
Pc{G) in a canonical multiplex ensemble is given by 



Pc(G) 



Zc 



cxp 



EWG) 



(14) 



where the normalization constant Zc is called the "parti- 
tion function " of the canonical multiplex ensemble. The 
values of the Lagrangian multipliers A M are determined 
by imposing the constraints given by Eq. (|12|) assuming 
for the probability Pc(G) the structural form given by 
Eq. (HI. 

In this ensemble, we can the relate the entropy S (given 
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by Eq. (|6])) to the canonical partition function Zc getting 
S = - p c(G) log Pc(G) 

G 

= -J2Pc(G)[-J2W(G) -bg(Zo)] 

G V 

= ^A^ + log^c- (15) 



the entropy of the multiplex can be used in solving infer- 
ence problems and is a first principle measure to quantify 
the complexity of the ensemble. In the following we give 
some example of uncorrelated and correlated canonical 
multiplex ensembles. 



B. Examples of uncorrelated canonical multiplex 
ensembles 



We call the entropy S of the canonical multiplex ensemble 
the Shannon entropy of the ensemble. 



1. Multiplex ensemble with given expected total number of 
links in each layer 



Uncorrelated or correlated canonical multiplex 
ensembles 



For a canonical uncorrelated multiplex ensemble in 
which each multiplex G has probability P{G), we have 
that Eq. Q is satisfied, i.e. 



M 



Pc(G) = J] Pg(G a ). 



(16) 



where Pg(G a ) is the probability of network G a on layer 
a. Given the structure of the probability Pc(G) in the 
canonical multiplex ensemble given by Eq. (|14[) , in order 
to have an uncorrelated multiplex the functions F^{G) 
should be equal to a linear combination of constraints 
ffj,,a(G a ) on the networks G a on a single layer a, i.e. 



M 



(17) 



a=l 



A special case of this type of constraints is when each 
constraint depends on a single network G a in a layer a. 
In this case typical sets of constraints can be: the average 
total number of link in each layer, the expected degree 
sequence in each layer, the expected degree sequence and 
the expected community structure in each layer etc. In- 
stead, in the case in which the multiplex is correlated, 
also quantities such as the expected overlap can be fixed. 
For a multiplex formed by two layers, we can therefore 
construct multiplex ensembles with expected total num- 
ber of links in each layer and expected global overlap be- 
tween the two layers, or with expected degree sequence 
and expected local overlap between the two layers etc. 

We can therefore construct a large class of canonical 
uncorrelated and correlated multiplex ensembles enforc- 
ing a different number of constraints. Starting with a 
minimal number of constraints, when we introduce fur- 
ther constraints in our ensemble we expect that the typ- 
ical number of multiplexes that satisfy the constraints 
will decrease, and therefore we expect that the entropy 
of the multiplex ensemble will decrease. Multiplex in net- 
work ensembles with smaller typical number of realiza- 
tions are more complex and more optimized. Therefore 



We can fix the average number of links in each layer 
a to be equal to L a . In this case we have K = AI 
constraints in the system indicated with a label a = 
1,2, ... ,M. These constraints are given by 

J2Fo t (G)P(G)^J2zZ a ^ P (G) = L a (18) 

G G *<J 

with a = 1, 2, . . . , M. Therefore the explicit expression 
for F a (G) is given by 



F a {G) = Y.<i 



(19) 



The probability of the multiplex is given by Eq. (fFfj) . Us- 
ing this expression we observe that the probability Pc (G) 
can be written as 



^c(G) = -^exp 



(20) 



where Zc is the canonical partition function and X a is 
the Lagrangian multiplier enforcing the constraint given 
by Eq. (|18p. The probability of a link between node i 
and node j in layer a is given by 



P 



P 



The Lagrangian multipliers are fixed by the condition 



(21) 



i<j 



(22) 



2L a /[N(N - 1)] and e 



2L° 



I.e. p — ^ -vj " N(N~l)-2L° 

Using the definition of the entropy of the multiplex Eq. 
© and the expression for Pc(G) given by Eq. (f2T?T) it is 
easy to show that the entropy of the canonical multiplex 
ensemble S, that we call Shannon entropy, is given by 



iV(7V - 1) 



J2lP a logp a + (l-p a )log(l-p a )]. 

(23) 



a=l 
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where p a = 2L a /[N(N - 1)]. If the number of layers M 
is finite, it can be shown that this expression in the large 
TV limit, is equal to 



M 



Af(iV-l) 

S = log ( 2 



a=l 



L' 



2. Multiplex ensemble with given expected degree sequence 
in each layer 

We can fix the expected degree kf of every node i in 
each layer a. In this case we have K = M x N constraints 
in the system indicated with a labels a = 1, 2, . . . , M and 
i = 1, 2 . . . , N. These constraints are given by 

N 

Y^F %a (G)P(G) = J2 E *%P(G) = K- (24) 

G G J=1.jV* 

Therefore the explicit expression for Fi tCi (G) is given by 



N 



Fi, a (G)= E 



(25) 



The probability of the multiplex is given by Eq. (fll)) . Us- 
ing this expression we observe that the probability Pc(G) 
can be written as 



Pc(G) = -^exp 



M N 



N 



-£I> Q e 



(26) 



where is the canonical partition function and Ai ;Q , is 
the Lagrangian multiplier enforcing the constraint given 
by Eq. The probability of a link between node i 

and node j in layer a is given by 



Pi 



= T- _ A _ A 



(27) 



where the Lagrangian multipliers Ai. Q are fixed by the 
conditions 



iV 



E P ij ~ 



(28) 



3 = 1,j¥=i 



Using the definition of the entropy of the multiplex Eq. 
© and the expression for Pc{G) given by Eq. (f2l>| it is 
easy to show that the entropy of the canonical multiplex 
ensemble S, that we call Shannon entropy, is given by 



m 



S=-J2 Z>3 lo §^ + ^ ~ P«) log(l - Pg)]. (29) 

a— 1 t<j 



If fcf < ^(k a )N Mi = 1, 2 . . . , JV then each network G" is 
uncorrelated and therefore, e _Ai >° " 



and 



k^k^ 

jj^jpj- In this limit the Shannon entropy 5 is given by 
p 



5* e 



1 



■ E *? log(*f ) + i(fc Q )/Vlog((fc«)/V) 



2 V 1 4 V (fc a ) 



(30) 



3. Multiplex ensemble with given expected number of links 
present in each layer between nodes in different communities 

We can fix the expected number of links present in 
each layer between nodes belonging to different commu- 
nities. We assign to each node i a discrete variable qi = 
1, 2, ...,Q indicating the community of the node. We 
can consider canonical uncorrelated multiplex ensembles 
in which we fix the expected number of links e q q , between 
nodes in community q and nodes in community q' in layer 
a. In this case we have K = M x Q(Q + l)/2 constraints 
in the system indicated with a labels a = 1, 2, . . . , M and 
q, q' = 1,2 ... ,Q. These constraints are given by 



E^'c (G)P(G) = e a q>q , 



(31) 



where the explicit expression for F g g i a (G) is given by 
F q ,q'A Ga ) = E'VV' 'V-'< • iorq^q' 
F q , q AG a ) = E'VVv'V; • (32) 



The probability of the multiplex is given by Eq. (|14[) . Us- 
ing this expression we observe that the probability Pq(G) 
can be written as 



Pc(G) = ^-exp 



M 



/ ] / ] ^q,q' ,aFq,q' ,a(G) 
a — 1 q<q' 



(33) 



where Zc is the canonical partition function and \q t q> , Q is 
the Lagrangian multiplier enforcing the constraint given 
by Eq. (f3"T|) . The probability of a link between node i 
and node j in layer a is given by 



(34) 



1 



where the Lagrangian multipliers are fixed by the condi- 
tions 



E^AsA'^ 

id 



eq. g > for 9 ^ q' 



q-qj 



(35) 



6 



As it can be seen by Eq. (|34|) the probabilities pfj depend 
only on Qi,qj and a therefore we have pfj = p a (qi,qj) 
with 



P a (q,q) = iovq^q' 
n q n q , 



P a (q,q) 



■'1,1 



n q( n q - !)/ 2 



(36) 



where n q indicates the total number of nodes in commu- 
nity q. Using the definition of the entropy of the multi- 
plex Eq. (|6]) and the expression for Pc{G) given by Eq. 
([33| it is easy to show that the entropy of the canonical 
multiplex ensemble S, that we call Shannon entropy, is 
given by 



M 

S=-J2 5>« l0 ^ + 0- - P?i) !og(l - Pii)\- (37) 

a—l i<j 

If the number of constraints is non extensive M x Q(Q + 
l)/2 <C N, this expression in the large N limit is given 
by 



M 



s 



a=l q=£q' 
a—l q 



where the explicit expression for Fi a (G) and for 
F q , q >, a {G) is given by 



JV 



FUG) = J2 a % 
F q . q -. a {G a ) = X^A-jA'^ - f o r 9 ^ 9 



'■./ 



Fq,q,a{G) — ^ ] a jjS q . qi S q , q 



(40) 



The probability of the multiplex is given by Eq. (fT4")) . Us- 
ing this expression we observe that the probability Pc (G) 
can be written as 



Pc(G) = ^exp 

AC 



x cxp 



M N 



a — l i—1 

M 



/ ] y ] ^q.q' .aF q , q > , a (G) 
a—l q<q' 



(41) 



where is the normalization factor, Ai iQ is the La- 
grangian multiplier enforcing the constraint given by Eq. 
(J3"5| and \ q ,q',a is the Lagrangian multiplier enforcing the 
constraint given by Eq. (j3"9")l . The probability of a link 
between node i and node j in layer a is given by 



1 + e 



(42) 



where the Lagrangian multipliers are fixed by the condi- 
tions 



4- Multiplex ensemble with given expected degree sequence 
in each layer and given expected number of links present in 
each layer between nodes in different communities 

We assign to each node i a label g,; = 1,2 ... ,Q indi- 
cating the community to which node i belongs. We can 
consider canonical uncorrelated multiplex ensembles in 
which we fix the expected degree kf of every node i in 
each layer a together with the expected number of links 
e i i' between nodes in community q and nodes in commu- 
nity q' in layer a. In this case we have M x N constraints 
in the system indicated with a labels a = 1,2, . . . ,M and 
i = 1, 2 . . . ,N and other constraints indicated 

with labels a = 1, 2, . . . , M and q, q' = 1, 2 . . . , Q. These 
constraints are given by 



N 

E p% = ^ 

3 = 1,3^ 

Z^P% 5 1,1< 5 l',1i = e q,q' ior 9 ¥= Q' 

X>?A«<W = e«, (43) 

i<j 

Using the definition of the entropy of the multiplex Eq. 
© and the expression for Pc(G) given by Eq. (|4"T1) it is 
easy to show that the entropy of the canonical multiplex 
ensemble S, that we call Shannon entropy, is given by 

S=- E5>S lo S^ + C 1 - Ptf) 1 °S0- ~ Pij)]- (44) 

a i<j 



X>, Q (G)P(G) = k\ 

G 

Y,Fq,q>,a(G)P(G) 



1,1 



C. Properties of the uncorrelated canonical 

(38) 

multiplex ensembles under consideration 

(39) In all the ensembles taken in consideration in the pre- 

vious subsection the existence of any link is independent 



7 



on the presence of other links in the multiplex and the 
probability of a given multiplex G is given by 



A I 



Pc(G) = Ul[[Pi 

a—1 i<Cj 



x« + (1 (45) 



Using the definition of the entropy of the multiplex Eq. 
© and the expression for Pc{G) given by Eq. (|4"5)l we 
can show that the entropy of the canonical multiplex en- 
semble S, that we call Shannon entropy, is given by 



A I 



S = -J2 T,K lo §^ + C 1 - P%) - Pii)\- ( 46 ) 



a— 1 i<j 



for all the cases under consideration in subsection \IV B\ 
In the considered ensembles we can calculate the av- 
erage global overlap (O a,a ) between two layers a and 

a' and the average local overlap (o"' a ) between two lay- 
ers a and a' where the global overlap O a - a is defined in 
Eq. Q and the local overlap o"' a is defined in Eq. (|5|). 
These quantities are given by 



i<j 



N 



PijPij 



(47) 



For a multiplex ensemble with fixed expected total 
number of links L a in each layer a we have pfj = p a = 
2L a /[N(N - 1)] and therefore, 



(O a ' a ) = 



2L a L a 
N(N — 1) 
4L a L a ' 
N 2 (N - 1) 



(48) 



Therefore if L a = O(N) Vet = 1, 2, ... , M, then the aver- 
age global overlap is a finite number in the large network 
limit and the local overlap is vanishing in the large net- 
work limit. Therefore in this case the overlap of links is a 
totally negligible phenomena in the multiplex. In fact the 
average global overlap (O a ' a ) is much smaller than the 
total number of links in layer a, L a or the total number 
of links in layer a' , i.e. L a . Moreover the average local 
overlap (o"' a ) is much smaller that the expected degree 
of node i in layer a or in layer a'. For multiplex en- 
sembles with given expected degree of the nodes in each 



layer, and with kf < \/ (k a )N we have p° 
therefore 



{k")N 



and 



(O a ' a 



I a. a. \ 



/ / foot foot V 



(k a ) (k a ') 
k a k a ' 



h a h a 



(k a ) (k a ')N 



(49) 



where (k a k a ' ') = kfkf/N. 

If the degrees in the different layers arc uncorrclatcd 



(i.e. (k a k a j = (k a ) 
overlaps are given by 

(O a - a ') = 



k a )) then the global and local 

(k a )(k a ' 



1 

2 

N 



< N 



< min(fcf,fcf ) 



(50) 



Therefore also in this case the overlap is negligible. De- 
gree correlation in between different layers can enhance 

the overlap, but as long as (k a k a ) -c N the average 



global (O q,q ) and the local (o™' a ) overlap continue to re- 
main negligible with respect to the total number of nodes 
in the two layers and the degrees of the node i in the two 
layers. Similarly using Eq. (|47|) it is possible to calculate 
the expected global overlap and local overlap also in the 
multiplex ensemble in which we fix the number of links 
that in layer connect nodes belonging to different com- 
munities and in the multiplex ensemble in which we fix 
at the same time the average degree of each node in each 
layer and the average number of links in between nodes 
of different communities at any given layer. In general 
if in a multiplex ensemble we want to have a given sig- 
nificant overlap we need to consider correlated multiplex 
ensembles. 



D. Construction of a uncorrelated multiplex in an 
uncorrelated canonical multiplex ensemble under 
consideration 

In all the cases taken into consideration in the previous 
subsections, the probability of a network G a on layer a is 
uncorrelated with the other networks in the other layers. 
In particular, the probability of a multiplex G can be 
written as in Eq. (|4"5"j) . 

Therefore in order to construct a multiplex in the 
canonical network ensembles it is sufficient to follow the 
following scheme 

• Calculate the probability pfj to have a link between 
node i and j in layer a. 

• For every pair of node i and j put a link in layer 
a with probability pf - . Do this for every layer a = 



1, 2, . . . , M independently. 



E. Examples of correlated canonical multiplex 
ensembles 



If the probability of a multiplex Pc(G) does not fac- 
torize into the probabilities Pg(G a ) of the networks in 
the different layers a of the multiplex, i.e. if 



Pc{G) ± Pg(G a ) 



(51) 
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the multiplex is correlated. In these ensembles the ex- 
istence of a link in one layer can be correlated with the 
existence of a link in another layer. For single networks, 
when we want to treat ensembles in which the links are 
correlated we need to make use of a parametrization that 
takes into account not only of single independent links 
but also of correlated set of links called subgraphs, such 
a triangles, triples, and so on [37], l38| . Similarly if we 
want to treat correlated multiplex, it is convenient to 
consider multilinks. In this way our multiplex is not any- 
more described by M adjacency matrices describing the 
networks at each multiplex layer, but the network is de- 
scribed by a much larger set of variables corresponding 
to correlated links, i.e. multilinks, and is fully character- 
ized by 2 M multiadjacency matrices. The simplest case 
of correlated multiplex ensemble is an ensemble in which 
we fix the expected total number of multilinks fh in the 
network defined in section liTl Starting from this example 
of correlated canonical multiplex ensemble we can gen- 
erate more refined models in which we fix the expected 
multidegree sequence k™ defined in section [77] or the ex- 
pected number of multilinks fh linking nodes of different 
communities etc. In the following we will describe in 
detail some of the more relevant examples of correlated 
canonical multiplex ensembles. 



1. Multiplex ensemble with given expected total number of 
multilinks fh 



We can fix the average number L m of multilinks fh 
with the condition ^-1™ = N(N - l)/2. In this 
case we have K = 2 constraints indicated by the label 
m = (mi, m2, . • • , m Q , . . . , uim) with m a = 0, 1. These 
constraints are given by 

E F rn (G)P C (G) = E E A t P c (G) = L* (52) 

G G *<3 

where the multiadjacency matrices of elements A J™ are 

defined in Eq. ([JJ. In this case the functions F^(G) are 
given by 



F^{G) = Y,Afi- 



(53) 



The probability Pc{G) of a multiplex in the ensemble is 
given by Eq. (fT4"]) that reads in this specific case 



P C (G) = ^exp 
Zc 



-E A ^E^ 



i<j 



(54) 



where Zc is the canonical partition function and is 
the Lagrangian multiplier enforcing the constraint given 
by Eq. (|52"1) . The probability p 7 ^ of a multilink fh be- 
tween node i and node j is given by 



,-A,55 



-A™ 



(55) 



with T,i<jPtj = L ™> and HfhPfj = 1 implying 

r fh 



V 



N(N — l)/2' 



(56) 



The entropy of the canonical multiplex ensemble S given 
by Eq.© can be calculated using the expression for 
Pc{G) Eq. (J5ll), obtaining 



S 



N(N - 1) 



Eulogy 5 



(57) 



with p m is given by Eq. (|56"|) . If the number of layers M 
is finite this entropy S is given by 



S = log 



N(N-1) \, 



(58) 



2. Multiplex ensemble with given expected multidegree 
sequence 

We can fix the average multidegree kf 1 of node i with 
the condition Ylm^T = N — 1. In this case we have 
K = 2 M x N constraints indicated by the label fh = 
(mi, ni2, . . . , m Q , . . . , uim) with m a =0,1 and the label 
i = 1, 2, . . . , N. In particular we have, 

E F hrn (G)P c (G) = E E A % P c(G) = kf (59) 

G G 3 

for all fh with m a =0,1 and alH = 1, 2, . . . N, where the 
multiadjacency matrices of elements = 0, 1 are given 

by Eq. dJJ). Therefore the functions F iy ^i(G) are given in 
this case by 



N 



*U(<3)= E A tr 

3=1 



(60) 



The probability of the multiplex is given by Eq. (| 14[) that 
in this case reads 



P(G) = — exp 



N 



N 



EE^E^ 



3 = 1 



(61) 



where Zc is the canonical partition function and Xi t m is 
the Lagrangian multiplier enforcing the constraint given 
by Eq. (|59|) . The probability of a multilink fh between 
node i and node j is given by 



(Af 3 ) = 



y _ e -Ai,«-Aj )ril 



(62) 



with the Lagrangian multipliers A.^,^ fixed by the con- 
straints 



E4 

■m 
N 

E4 

3=1 



(63) 



The entropy of the canonical multiplex ensemble S, that 
we call Shannon entropy, is given by Eq.© and can be 
calculated using the expression for Pc(G) Eq. (jrTTj) . ob- 
taining 

^ = -EE^ lo §4]- ( 64 ) 

in i<j 

If the multiplex is sparse, i.e. fc™ < \J (k"" l )N pro- 
vided that in the multilink rh there is at least a link, 

i.e. Yla=i TO « > 0' we navc 



(k" l )N 



(65) 



for all m such that X)a=i m « > 0. In this limit the 
entropy S is given by 



S 



E 



log(fcf) + i(fc™)7V 



+ I( fc -)iVlog((^)iV)-i(i^ 



(66) 



where the explicit expression for F q>q i ,rn(G) is given by 
F M >,ih{G) = Ys A ti 5 i^ 5 i'>iji ^>xq + q' 



Fq,q,m(G) — /.Af-Sg^S, 



(68) 



><J 



and the multiadjacency matrices of elements A™ are de- 
fined in Eq. ([1]) . The probability of the multiplex is given 
by Eq. (|14[) and in this specific case is given by 



P C (G) = — exp 



y ] y ] ^q.q' ,fn,Fq,q' .m(G) 



(69) 



where Zc is the canonical partition function and \ q ^ q i ^ 
is the Lagrangian multiplier enforcing the constraint 
given by Eq. (|67p . The probability of a multilink rh 
between node i and node j is given by 



Pi 



(At 



(70) 



where the Lagrangian multipliers are fixed by the condi- 
tions 

E4 = 1 

m 



q- q j 



(71) 



s<3 



As it can be seen by Eq. ([70)) the probabilities depend 

only on qi,qj and rh therefore we have = p m (qi,qj) 
with 



3. Multiplex ensemble with given expected number of 
multilinks m between nodes in different communities 

We can fix the expected number of multilinks rh 
between nodes in different communities of the multi- 
plex. We assign to each node i a discrete variable 
qi = 1, 2, . . . , Q indicating the community of the node. 

We can consider canonical uncorrelated multiplex en- 
sembles in which we fix the expected number of multilinks 
m, e™ 9 / between nodes in community q and nodes in com- 
munity q' . Moreover we choose e™ g / such that they sat- 
isfy the condition that the sum over the different multi- 
links rh of e™ ?/ is equal to the total number of links in be- 
tween nodes in community q and nodes in community q' . 
In this case we have K = 2 x Q(Q + 1)/2 constraints in 
the system indicated with a labels rh = (mi, m,2, . . . , ttim 
with m a — 0, 1 and the labels q, q' = 1,2,... Q. These 
constraints are given by 



P rn (q,q) 



c q,q' 

T*qTlq' 



for q 7^ q' 



n q (n q - l)/2 



(72) 



where n q indicates the total number of nodes in commu- 
nity q. The entropy of the canonical multiplex ensemble 
S that we call Shannon entropy is given by Eq. ([6]). 
Evaluating this expression using the probability of the 
multiplex Pc{G) given by ([69]) we obtain, 



s=-EE^ lo s4i- 



(73) 



If the number of constraints is non extensive 2 M Q(Q + 
l)/2 <C N, this expression in the large N limit is given 
by 



S = £log 



{n q n q/ )\ 

Y\m( e q,q' 



+ 



n„(n„-l) \ 1 
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4- Multiplex ensemble with fixed expected multidegree 
sequence and expected number of multilinks rh between nodes 
in different communities 

We assign to each node i a label % = 1,2 ... ,Q in- 
dicating the community to which node i belongs. We 
can consider canonical uncorrelated multiplex ensembles 
in which we fix the expected multidegree kf l of every 
node i (with the condition k™ = N — 1) together 



with the expected number of multilinks 



between 



nodes in community q and nodes in community q' (with 
the condition that the sum over the different multilinks 
rfi of eT 1 , is equal to the total number of links in be- 
tween nodes in community q and nodes in community 
q 1 ). In this case we have 2 M x N constraints indicated 
with a labels rh = (mi, mi, . . . ,771m) with m a = 0,1 
and i = 1,2 ... ,N and other 2 M x constraints 
indicated with labels rh and q,q' = 1,2..., Q. These 
constraints are given by 



G 

J2 F ^'^nPc(G) = e™, lorq^q' 

G 

J2 F ^rn (G)Pc(G) = e™, 



(75) 
(76) 
(77) 



where the explicit expression for Fi^ r %(G) and for 
Fq.q'.miG) are given by 

Fi, a (G) = E4? 
j 

F q , q >MG) = Y^ A t 5 i,iA',iv for 9 7^' 
F q>q>A {G) = X A Pw <Vv ■ (78) 



i<j 



where the element A™ of the multiadjacency matrices is 
defined in Eq. ([1]). The probability of the multiplex is 
given by Eq. (|T4")) that reads in this case 



Pc(G) 



Zc 



x exp 



exp -^^X l ,rnF l .ff l (G) 

m i 

^ ] ^ ] Xq,q'mFq,q'm{G) 
m q<q' 



(79) 



where Zq is the canonical partition function, is the 
Lagrangian multiplier enforcing the constraint given by 
Eq. (|75[) and \ q>q ' t m is the Lagrangian multiplier enforc- 
ing the constraint given by Eq. (|76j) or by Eq. (|77|) . The 
probability of a multilink rh between node i and node j 
is given by 



(80) 



where the Lagrangian multipliers are fixed by the condi- 
tions 



E4 = 1 

m 

E4 = fc f 

E^'/V'/ ^ 



e™ , for g ^ g' 



9-9, 



(81) 



The entropy of the canonical multiplex ensemble that we 
call Shannon entropy is given by 



(82) 



m i<3 



where the probabilities p™j are given by Eq. ([50)1 and 
satisfy Eqs. ([ST]). 



F. Overlap in correlated canonical ensembles under 
consideration 

In all the cases taken into consideration in the previous 
subsection, the probability of a network G a on layer a is 
correlated with the other networks in the other layers. 
Therefore the probability Pc(G) cannot be factorized 
in the probability for single layers. Nevertheless Pc(G) 
takes a simple form in the cases that we have investigated 
so far, i.e. 



Pg(G) = Y[ 



i<j 



nrf 1 A 



(83) 



where rh = (mi, mi, . . . , tum) is a vector of elements 
m a — 0, 1 and where A"j are the multiadjacency matri- 
ces defined in Eq. ([I}. In these ensembles the Shannon 
entropy S given by Eq. (J6j) takes the simple form 



s = -EE^ 10 ^ 



(84) 



i<j rh 



In the considered ensembles we can calculate the average 
total overlap (O a ' a ) between two layers a and a' and the 

average local overlap (c\' a ) between two layers a and 
a' , where the global overlap O a,a is defined in Eq. (HJ 

and the local overlap c\ ' a is defined in Eq. (JSJ. These 
quantities are given by 



(o a - a ') = E E4 

m\m a = l,m a i=l i<j 
N 



a, a 



O; 



E £4- 

m|m a -l,m Q /-l j— 1 



(85) 
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These quantities now can be significant also for sparse 
networks as we will see in the next subsection in the 
simple case of a multiplex with just two layers, i.e. M = 
2. 



G. Case of a two layers multiplex, i.e. M — 2 

Let us consider the simple case of a correlated multi- 
plex ensembles formed by M = 2 layers, network 1 and 
network 2. The probability Pc{G) of a multiplex in all 
the cases taken in consideration in the subsection \IV El 
is given by Eq. (|55|) that reads in this case 



p(G) = n 



4) + P«4( 1 



l<3 



+p°i(i-4)4 + 



■ Pi/44] 



(86) 



where p™ 1 '™ 2 is the probability to have ni = 0, 1 links be- 
tween node i and node j in network 1 and n% = 0, 1 links 
between the same nodes in network 2. The probabilities 
p^ 1 " 2 satisfy the constrain p°° , 
entropy of such multiplex is then given by Eq. 
reads in this case 



p°J+pl? +pjj = 1. The 
Ml) that 



(87) 



In the considered ensembles we can calculate the av- 
erage total overlap (O 1 ' 2 ) = (O) between two layers 1 



and 2 and the average local overlap (o i ' ) = (oi) defined 
in Eqs. ((55)) . For the ensembles in which we fix the ex- 
pected total number of multilinks m, L considered in 
subsection \IV E Tl we have 



(O) = L 



ii 



2L 11 
iV-1' 



Assuming i u ,L 10 ,L 01 cx JV, Eq. (JSSJ implies that the 
fraction of links that overlap is not negligible (globally 
and locally) also if both network 1 and network 2 are 
sparse. For the ensemble in which we fix the expected 
multidegree (considered in subsection IIV E 2]) . consider- 
ing the additional condition fc™ < y/ (A; 7 ™) AT for all mul- 
tilinks m formed at least by a link, i.e. X)a=i TO « 0, 
we have p$ = ,,'^^ r and therefore, 



<fc™)7V 



(O) 



1 



(fc n )iV 



(89) 



Provided that (fc 11 ) is finite, we find that also in this case 
the global and local overlap can be significant also if both 
network 1 and network 2 are sparse. A similar conclusion 
can be drawn for the other two cases of correlated mul- 
tiplex ensembles taken in consideration in the previous 
paragraphs. 



H. Construction of correlated multiplex in the 
canonical multiplex ensemble 

Since in the considered cases of correlated multiplex 
ensemble the probability of a multiplex can be expressed 
as in Eq. (|83|) . in order to construct a correlated multi- 
plex in the canonical network ensembles it is sufficient to 
follow the following scheme. 



• Calculate the probability p^ 
between node i and j. 



to have a multilink 



• For every pair of node i and j, draw a multilink 
m with probability pYj and consequently put a link 
in every layer a where m a = 1 and put no link in 
every layer a where m a = 0. 



V. MICRO-CANONICAL MULTIPLEX 
ENSEMBLES 

The microcanonical multiplex ensembles are formed by 
the multiplexes that satisfy some hard constraints. Ev- 
ery multiplex in a microcanonical multiplex ensemble has 
equal probability. We note here that we consider only 
graphical constraints [45| . i.e. constraints that can be 
satisfied at least by one realization of the multiplex. This 
is a condition that for example is automatically satisfied 
if we consider network ensembles that are a randomiza- 
tion of a real multiplex with some given structural fea- 
tures. Therefore the probability Pm(G) of a microcanon- 
ical multiplex ensemble is given by 



1 



PM(G) = — X[5[F li {G),C, 



(90) 



where <5[] is the Kronecker delta and where Zm is the "mi- 
crocanonical partition function" of the multiplex given 
by 



(91) 



G A*=l 



Therefore the microcanonical partition function Zm of 
the multiplex ensemble counts the number of multi- 
plexes satisfying the hard constraints F^(G) — C M for 
/x = 1,2 ... ,P. We call the entropy of these multiplex 
ensembles NT, and using the definition of the entropy of 
an ensemble given by Eq.® together with the expression 
for the probability of a multiplex in the microcanonical 
ensemble given by Eq. (|90j) we have 



(92) 



NT = - Pm (G) log P M (G) = log Z M , 



where we call E the Gibbs entropy of the multiplex en- 
semble. The Gibbs entropy £ of microcanonical multi- 
plex ensembles is related to the Shannon entropy S of the 
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associated canonical multiplex ensemble S which enforce 
the same constraint of the microcanonical network en- 
semble in average (the conjugated canonical ensemble), 
by a simple relation. In fact we have 



iVS = S - NCI 



(93) 



where NCI is equal to the logarithm of the probability 
that in the conjugated canonical multiplex ensemble the 
hard constraints F^(G) are satisfied, i.e. 



M 



Nil = - log \ J2 P c(G) I] C »\ > ■ 

G ^=1 



In order to verify the relation Eq. (|93|l we observe that 
the canonical multiplex probability Pc(G) is given by Eq. 
(1141) that we rewrite here for convenience, 



Pc(G) = 



1 



(95) 



and therefore, using Eq. we get 



exp[-iVO] = ^le-^V^ f[6[F^3),C^) 

G ° M=l 



1 

Zm 



G M=l 



exp[AT£-S]. 



(96) 



where in the last relation we have used Eq. (fTSj) . Eq. (|9"Tj) 
and Eq. (|92|) . Given Eq. (|93| . if ft is larger than zero in 
the limit N 1, the microcanonical and the conjugated 
canonical multiplex ensemble are not equivalent. 



A. Uncorrelated microcanonical multiplex 
ensembles 



Given the condition Eq. (|97)) the Gibbs entropy £ of 
the multiplex can be expressed as in the following 

NZ = -Y,PM(G)logP M (G) = Y,NX a . (98) 

G a 

where S Q is the Gibbs entropy of the network ensemble 
induced in layer a, 



iVS Q = - J2 P M(G a ) log P^(G a ) 



(99) 



with P^(G a ) = UJ[F-AG a ),C^ a ]/Z^ and 

zti = x) n 5 ^«( Ga )' ^.j- ( io °) 



Using the same arguments used to derive Eq. (|93)) it is 
straightforward to show that the Gibbs entropy S Q of 
each network ensemble at layer a is given by 



ivs Q = s a - nq c 



(101) 



where S a is the Shannon entropy of the canonical net- 
work ensemble which enforce the same constraint of the 
microcanonical network ensemble in average, i.e. 

S a = -J2 P S(G a )^gPg(G a ), (102) 

where Pg(G a ) is the probability for a network G a in 
layer a. Moreover Cl a in Eq. (|101[) satisfies 

Nn a = ~ log ^2 p c(° a ) II 5 l F »AG a ), Cu, a ] I . 

(103) 

Examples of uncorrelated microcanonical multiplex en- 
semble are given by ensembles in which we fix the total 
number of links at each layer, the degree sequence at each 
layer, the number of links between nodes in different com- 
munities in each layer etc. In the following subsection we 
present in detail several examples of uncorrelated micro- 
canonical multiplex ensembles. 



In an uncorrelated multiplex ensemble we have that 
the probability of a multiplex G is factorizable into the 
product of probabilities P a (G a ) of the networks G a in 
layer a, i.e. 

M 

Pm{G) = II P M(G a ). (97) 

a=l 

Given the general expression for Pm(G) provided by 
Eq. (|9U1) we can conclude that a microcanonical multi- 
plex ensemble is uncorrelated only if the hard constraints 
F^(G) — Cfj, with /i = 1,2 ... ,K involve for every con- 
straint fi only one network G a in one layer a of the multi- 
plex. Therefore we will indicate the function F^{G) with 
a label indicating the layer a and one label v counting 
the number of constraints in each layer, i.e. F v ^ a (G a ). 



B. Examples of uncorrelated microcanonical 
multiplex ensembles 

1. Multiplex ensemble with given total number of links in 
each layer 

We can fix the total number of links L a in each layer a 
of the multiplex. In this case we have K = M constraints 
in the system indicated with a label a = 1,2, ... ,M. 
These constraints are given by 

F a {G) = L a (104) 

with a = 1, 2, . . . , M and with F a {G) given by 

F a {G)=Y j a%- (105) 

i<j 
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The microcanonical partition function Zm is equal to 
the number of multiplexes in these ensemble, which is 
given by the product over the layers a = 1,2..., M 
of the number of networks G a satisfying the constraints 
F a (G) = L a . The number of networks G a with L a links 
is given by the number of ways of choosing L a links out 
of N(N — l)/2 possible links, we have therefore 




(106) 



Using Eq. (|92|) wc find that the Gibbs entropy for this 
ensemble is given by 



NT. 




(107) 



As long as the number of constraints M is sublinear with 
respect to N we have that the microcanonical and canon- 
ical ensemble studied in subsection \IV _6"Tl are equivalent 
in the thermodynamic limit and £ ~ S/N. 



2. Multiplex ensemble with given degree sequence in each 
layer 

We can fix the the degree kf of every node i in each 
layer a. In this case we have K = M x N constraints in 
the system indicated with a labels a = 1, 2, . . . , M and 
i = 1,2..., N. These constraints are given by 



F i>a (G) = kf. 



with Fi <a (G) given by 



2V 



0=1 



(108) 



(109) 



For this ensemble we can use the results of [33j,l35( getting 



NT = S — Nfl 



(110) 



with S given by Eq. (|2"i?|) and NCI for sparse networks is 
given by 



M N 



an) 



a—l i—1 



where Tt y (x) is the Poisson distribution with average y 
Tr y (x) = \/x\y x exp[— y]. In this case, if the number of 
layers M is finite, then in the large network limit TV 3> 1, 
O is finite, and we have T = S/N — £1. Therefore the 
Gibbs entropy T is lower than S/N and the microcanoni- 
cal ensemble is not equivalent in the thermodynamic limit 
TV ^> 1 to the co njugated canonical ensemble. In the case 
in which kf < y/(k a )N we can use for S the expression 



in Eq. (|30|) . Therefore the Gibbs entropy T can be ap- 
proximated by 



M 



NT = Y, log 



a=l 



Jilt")*) 

((k a )N)U 4 W 

M e 



(112) 



This last expression is a generalization of the Bender for- 
mula [3~il . |46| for the entropy of networks with given de- 
gree sequence. 



3. Multiplex ensemble with given number of links in each 
layer between nodes of different communities 

We can fix the total number of links between nodes of 
different communities in each layer a. We assign to each 
node i a discrete variable qi = 1,2, ... ,Q indicating the 
community of the node. Wc consider a microcanonical 
uncorrelated multiplex ensemble in which we fix the to- 
tal number of links , between nodes in community q 
and nodes in community q' in layer a. In this case we 
have K = MxQ(Q + l)/2 constraints in the system indi- 
cated with a labels a = 1,2, ... ,M and q,q' = 1,2 ... ,Q. 
These constraints are given by 

F q>q , a {G) = el ql (113) 

where explicit expression for F q q i a (G) is given by 

Fq,q>,a{G) = ]P agtf,,,, 6 q > , qj , for q ^ q' 

F q , q , a (G) = »'Av<V/ • (H4) 



The microcanonical partition function Zm is equal to 
the number of multiplexes in this ensemble, which is 
given by the product over the layers a = 1,2 ... ,M 
of the number of networks G a satisfying the constraints 
Fq, q >, a (G) = e qq >- The number of networks G a with e qql 
links is given by the number of ways of choosing e q , links 
out of the total number of possible links between nodes 
in community q and community q' , wc have, therefore, 



n 



n 

q<q' 



(n q n q >)\ -q (n q {n q - l)/2)! 



a I 
'9,9" 



(115) 



where n q indicates the number of nodes in community q. 
Finally the Gibbs entropy T for this ensemble is given by 
Eq. (pJTj) and therefore we obtain 



M 



NT = J2 log 



n 



(n q n q >)\ -r-r {n q (n q - l)/2)! 



Lq 1 iq< 



n 



q<q> lil q 

(116) 

In this case the Gibbs entropy T = S/N in the limit 
N I only if the number of constraints P is sublinear 
with respect to N. 
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jf.. Multiplex ensemble with given degree sequence in each 
layer and given number of links in between nodes in different 
communities in each layer 

We assign to each node i a label qi = 1,2 ... ,Q in- 
dicating the community to which node i belongs. Wc 
can consider microcanonical uncorrelated multiplex en- 
semble in which we fix the degree kf of every node i in 
every layer a together with the total number of links efjq, 
between nodes in community q and nodes in community 
q 1 in layer a. In this case we have M x N constraints in 
the system indicated with a labels a = 1,2, ... ,M and 
i = 1,2 ... ,N and other M ® g constraints indicated 
with labels a = 1,2, ... ,M and q,q' = 1,2 ... ,Q. These 
constraints are given by 



Fi, a (G) = kf 

q,q',a(G) — E 



(117) 



where the explicit expression for Fi. a (G) and for 
F q ,q',a(G) is given by 



JV 

FUG) = £ < 
F q , q >, a (G a ) = ^afjSq^Sq,^, for q £ q' 
Fq.q. a (G) = ^ ] a ij^q,qi^q,qj > 

i<j 

The Gibbs entropy for this ensemble satisfies 

ivs = s-^nci", 



(118) 



(119) 



where S is given by Eq. (|4"4"f and using the results of [35[ 
the entropy of large variations tt a for sparse networks is 
given by 



/v 



Mr = -J> g [**-(*?)] -£>g 

4—1 



(120) 

where Tr y (x) is the Poisson distribution with average y 
given by ir y (x) = — } y x exp[— y\. In this case, if the num- 
ber of layers M is finite, then in the large network limit 
N > 1, Cl is finite, and we have S = S/N - Q. Therefore 
the Gibbs entropy £ is lower than S/N and the micro- 
canonical ensemble is not equivalent in the thermody- 
namic limit to the conjugated canonical ensemble. 



5. Multiplex with given degree-degree correlations in each 
layer a 

We can construct a microcanonical uncorrelated mul- 
tiplex ensemble with given degree-degree correlations in 
each layer a by fixing the degree kf of each node i in layer 



a and the total number of links ef. k , between nodes of 
degree k and degree k in layer a. This case is a small 
modification of the previous case in which for every dif- 
ferent layer we identify a community of nodes at a given 
layer a as the set of nodes with given degree, i.e. qi = kf. 
The Gibbs entropy £ satisfies 



M 



A I 



NZ = s a - Nn ° ■ 



(121) 



Using the results of [35[ the entropy of large variations 
f2 a for sparse networks is given by 



JV 



Mr = -£log [n k? (kf)] - £log 



k<k' 



Kq') 



(122) 

Moreover the Shannon entropy S a for each layer a is 
given by 



with 

— Ai.c, —Xj.a —Afc.fc'.o 

(124) 



Pa = 



1 + e 



— A, „-Aj „-A 



and the Lagrangian multipliers Ai jQ , and Xk,k',a fixed by 
the conditions 



JV 



2J p§ - 

^pfjhf,kSk°-,k = e% >k . 

i<j 



(125) 



C. Correlated microcanonical multiplex ensembles 

In a correlated multiplex ensemble we have that the 
probability of a multiplex G is not factorizable into the 
product of probabilities P a (G a ) of the networks G a in 
layer a, i.e. 



M 



Pm(G) ± J] P lAG a ). 



(126) 



The simplest example of correlated multiplex ensemble is 
the ensemble in which we fix the total number of multi- 
links m in the multiplex. Starting from this model differ- 
ent more refined multiplex ensemble can be determined, 
fixing for example the multidegree sequence or the to- 
tal number of multilinks m in between nodes of different 
communities etc.. In subsection \V D\ wc will discuss in 
detail some relevant examples of correlated multiplex en- 
sembles. 
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D. Examples of correlated microcanonical 
ensembles 

1. Multiplex ensemble with given total number of multilinks 

m 

In a correlated multiplex ensemble we can fix the total 
number L m of multilinks to in the multiplex, i.e. 



F rn (G) = V 



(127) 



for all to = (mi, rri2, ■ ■ ■ , m a , ■ • ■ , tum) with m a = 0, 1, as 
long as J2m L m = N(N—l)/2. In this case the functions 
F,^(G) are given by 



Frn{G) 



i<j 



(128) 



where the multiadjacency matrices of elements A^/j are 
defined as in Eq. ([1} . Since any pair of nodes is linked by 
one multilink rh, we have the total number of multiplexes 
Zm in this ensemble is given by the multinomial 



N 



U,r,L 



(129) 



Using this result, we can easily derive the Gibbs entropy 
7V£ = log(Z M ),i.e. 



NY = log 



N 



Ti m L 



m | 



(130) 



As long as the number of constraints K = 2 M is sub- 
linear with respect to N we have that the microcanonical 
and the conjugated canonical ensemble are equivalent in 
the thermodynamic limit A" 3> 1 and E ~ S/N. 



2. Multiplex ensemble with given multidegree sequence 

In a correlated multiplex ensemble we can fix the mul- 
tidegree k™ of node i, 

Fi,m{G) = kf (131) 

for all to with m a =0,1 and all i = 1, 2, . . . N as long 
as J2mk" 1 = A^ — 1 and the constraints are graphical. In 

this case we have that Fi^G) is given by 

N 

F h m(G)= E A ?i> ( 132 ) 

where the multiadjacency matrices of elements A™ = 0, 1 
are given by Eq. (JXJ) - The Gibbs entropy E of this en- 
semble satisfies Eq. (|101[) that we rewrite here for con- 
venience 



AT E = S - Nn 



(133) 



with S given by Eq. 
the one reported in [32 



Using a similar derivation as 
1351 it is possible to prove that 



for sparse networks f2 is given by 



N 



]T 5>g7T fc j»(fcf) 



(134) 



where Tr y (x) is the Poisson distribution with average y 
TTy(x) — -^y x exp[— y] calculated at x. In this case, if 
the number of layers M is finite, then in the large net- 
work limit N 1, is finite, and we have E = S/N — £1. 
Therefore the Gibbs entropy E is lower than S/N and the 
microcanonical ensemble is not equivalent in the thermo- 
dynamic limit to the conjugated canonical ensemble. 
For networks with k\ 

= 1 



< y (k ,7l )N where rh satisfy 

the inequality X^«=i m a > 0> using Eq. ([66]) we can 
find a simple expression for the Gibbs entropy extending 
Bender result [3J, H(| to correlated multiplex, i.e. 



NY = log 



n 



(2L 



n lli=l ■ 



(135) 



3. Multiplex ensemble with given number of multilinks rh in 
between nodes of different communities 

We assign to each node i a label qi = 1, 2 . . . , Q indicat- 
ing the community to which node i belongs. We consider 
a microcanonical correlated multiplex ensemble in which 
we fix the total number of multilinks to, e™ g , between 
nodes in community q and nodes in community q' with 
the condition that the constraint is graphical. In this 



case we have 2 



M x Q(Q+i) 



constraints indicated with la- 



bels to = (toi, TO2, . . . , m a , . . . , Tom) with m a = 0, 1 and 



q,q 



1,2 ... ,Q. The constraints are given by 

F qA ' M (G) = J2 4X,A',fc- = 4,' for i * ?' 

F q>q> rh{G) = E4V^=4- ( 136 ) 



i<j 



For every pair of nodes, one in community q and one 
in community q' we will have one multilink to, therefore 
the total number of multiplex in this ensemble is given 
by Zm that has the explicit expression 



Z 



M 



{n q n ql )\ -pr {n q (n q - l)/2)! 



n r 

q<q i 11m q,q' 



n 



n „m | 
m q,q' 



(137) 



where n q is the number of nodes in community q. Finally 
the Gibbs entropy E of this ensemble, with A^E = log Zm 
satisfies 



NY, = log 



n jn q n q ,)\ -pj- (n q (n q - l)/2)! 
n f ™ ill 

1 lm g,<? 



<?<<?' 



n e fh | 
rh ^q,q' 



(138) 



16 



As long as the number of constraints P is sublincar with VI. CONCLUSIONS 

respect to N we have that the microcanonical and canon- 
ical ensemble arc equivalent in the thermodynamic limit 
and E ~ S/N. 



4- Multiplex ensemble with given multidegree sequence and 
given number of multilinks in between nodes of different 
communities 



We assign to each node i a label qi = 1,2 ... ,Q in- 
dicating the community to which node i belongs. We 
can consider a microcanonical correlated multiplex en- 
semble in which we fix the multidegree fc™ of every node 
i together with the total number of multilinks e™ q , be- 
tween nodes in community q and nodes in community 
q' with the condition that the constraints are graphi- 
cal. In this case we have 2 x N constraints indicated 
with a labels m = (mi, m,2, ■ ■ ■ , mm) with m a =0,1 
and i = 1, 2 . . . , N and other 2 M x constraints 
indicated with labels m and q,q' = 1,2 ... ,Q. The con- 
straints are given by 



JY 



\m _ ljm 



FiMG) = E 4 - 

^ 9 vn(G) = E4<w<v, 9 = for i * i' 



Fq,q,m(G) ^ ] ^-jj^q^qi^q^qj ~ S q,q' 



(139) 



The Gibbs entropy S of this ensemble satisfies 

NT, = S - NQ (140) 



where S is given by E q. (1821) and by following arguments 
similar to the ones in [35| it can be proved that for sparse 
networks fl satisfies the following relation 



M 



Nil 



E E ^ 

«=1 m|E"=i™ Q >0 

E E lo s 

9<9' m \ Y."=i m a >0 



7T fcf (fef) 

7r e * (e*,)l -(141) 



where K y (x) is the Poisson distribution with average y 
7r y( x ) = ~ti.1) x ex P[ — u] calculated at x. In this case, if 
the number of constraints P oc N, then in the large net- 
work limit N 3> 1, ^ is finite, and we have E = S/N — fi. 
Therefore the Gibbs entropy E is lower than S/N and the 
microcanonical ensemble is not equivalent in the thermo- 
dynamic limit to the conjugated canonical ensemble. 



In conclusion, we have presented a statistical mechan- 
ics approach for microcanonical and canonical multiplex 
ensembles. We have defined both uncorrelated and cor- 
related multiplex ensembles. Uncorrelated multiplex en- 
sembles are characterized by a probability of the multi- 
plex that factorize into the probability of the networks 
G a for every layer a of the multiplex. Therefore for un- 
correlated multiplex ensemble the probability a link in 
one network is independent on the presence of other links 
in the other layers. We have considered uncorrelated net- 
works in which we fix the expected number of links in 
each layer, the expected degree sequence in each layer, 
the expected number of links in between different com- 
munities in each layer, or the expected degree sequence 
and the expected total number of links between commu- 
nities in each layer. These ensembles, when describing 
multiplexes formed by sparse networks, have negligible 
global and local overlap, therefore they cannot model sit- 
uations in which the overlap of links in different layers is 
significant. In order to describe the situation in which 
the overlap is significant we introduced canonical corre- 
lated multiplex ensembles in which we fix the expected 
number of multilinks m given by L m , or the expected 
multidegree h™ sequence, or the expected number of mul- 
tilinks fh between nodes in different communities, or even 
expected multidegree sequence and expected number of 
multilinks between nodes of different communities. Fi- 
nally we characterize both microcanonical uncorrelated 
and correlated networks showing that the microcanon- 
ical ensembles and canonical ensembles are not equiva- 
lent as long as the number of constraints is extensive. 
This paper open a new scenario for studying multiplex 
ensembles and characterize null models of multiplex in- 
cluding a significant global or local overlap of the links 
in the different layers. In future works we plan to ex- 
tend this statistical mechanics of multiplex ensembles to 
more complex situations such as to directed and weighted 
networks, and to apply the entropy of multiplex for ex- 
tracting in formation from multiplex datasets. Moreover, 
recently new entropy measures for quantifying complex- 
ity of complex networks have been pro posed using tools 
of quantum in formation theory [47l. |48| . In future works 
we plan to generalize also these measures to multiplexes 
and use these new measure to uncover hidden statistical 
features of multiplex datasets. 
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